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Abstract 

Suppose that Xi, X2, . . . , X„ are independent and identically Bernoulli (6') distributed. 
Also suppose that our aim is to find an exact confidence interval for 9 that is the inter- 
section of a 1 — a;/2 upper confidence interval and a 1 — a;/2 lower confidence interval. 
The Clopper-Pearson interval is the standard such confidence interval for ^, which is 
widely used in practice. We consider the randomized confidence interval of Stevens, 
1950 and present some extensions, including pseudorandomized confidence intervals. We 
also consider the "data-randomized" confidence interval of Korn, 1987 and point out 
some additional attractive features of this interval. We also contribute to the discussion 
about the practical use of such confidence intervals. 
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1. Introduction 

Suppose that Xi,X2, . . . ,X„ are independent and identically distributed (iid), each 
with a Bernouni(6') distribution (6* G [0,1]). Let X = (Xi,...,X„). Our objective 
is to find a confidence interval for 9 of the form [^(X, V),u{X, V)~\ , where the interval 
endpoints may depend on an auxiliary random variable V, such that both of the following 
conditions are satisfied: 

ry 

Pg{9 > m(X, V)) < - for all 9 (1) 
Pe{9 < i{X, V)) < - for all 9. (2) 

Of course, such a confidence interval has infimum coverage probability that is greater 
than or equal to 1 — a. The conditions ([1]) and ([2]) make the endpoints of the confidence 
interval, i{X, V) and u{X, V), easy to interpret. Confidence intervals that satisfy these 
conditions are the discrete-data analogue of an equi-tailed confidence interval based on 
continuous data. The solution favoured by statistical practitioners is to find a non- 
randomized confidence interval for 9 based on Y = Xi + X2 + ■ ■ ■ + which has a 
Binomial(?T,, 9) distribution. The resulting Clopper-Pearson interval (Clopper and Pear- 
son, 1934) is widely used in practice. Of course, if the conditions ([1]) and ([2]) were to be 
replaced by the less stringent requirement that Pg(^i{X, V) < 9 < u{X, V)) > 1 — a for 
all 9 then other non-randomized confidence intervals such as that of Blaker (2000) would 
come into consideration. Nonetheless, there is still a lively interest in randomized and 
related confidence intervals, as evidenced by e.g. Geyer and Meeden (2005) and the re- 
sulting published comments. In the present paper, we will compare various randomized, 
"pseudorandomized" and "data-randomized" confidence intervals that satisfy ([T]) and ([2]) 
with the Clopper-Pearson confidence interval (described, for the reader's convenience, in 
Section 2). 

A randomized confidence interval can be found by considering the artificial data 
Z = Y + V , where V and Y are independent random variables and V has a uniform 
distribution on (0, 1) (Stevens, 1950). Equi-tailed 1 — a confidence intervals based on Z 
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dominate the 1 — a Clopper-Pearson confidence intervals. In Section 3, we review this 
randomized confidence interval and introduce some extensions to the idea of a randomized 
confidence interval, including pseudorandomized confidence intervals. In Section 4, we 
review the usual objections to the use of randomized confidence intervals in practice and 
note a new objection based on the need to condition on an ancillary statistic. 

Korn (1987) introduced a "data-randomized" confidence interval for 9 that uses the 
data itself to generate the randomization. This confidence interval does not require 
the use of an auxiliary variable V and overcomes some of the objections to the use of 
randomized confidence intervals in practice. In Section 5, we review this confidence 
interval and point out some additional attractive features of this interval. In Section 6, 
we note the objections of Senn (2007ab) to the use of such intervals in practice and note 
a further objection based on an invariance argument. 

In Section 7, we consider the properties of an unusual confidence interval for 6 that 
turns out to be a "data-randomized" confidence interval. We also explain why we expect 
this confidence interval to have properties that are inferior to the "data-randomized" 
confidence interval of Korn (1987). 

2. Clopper-Pearson confidence interval for the binomial probability 

Let fg{y) = Pg{Y = y) and Fe{y) = PeiX < y). For observed value y of F, the 
Clopper-Pearson 1 — a confidence interval for 9 is found as follows. The p-valuc for 
testing the null hypothesis Hq : 9 = 9 against the alternative hypothesis : 9 < 9 is 
P§{y ^ y)- A 1 — a upper confidence interval for ^ is {9 : Pe{Y < y) > q;/2}. We 
replace all upper confidence intervals of the form [6, c) by [6, c] . This does not decrease 
the coverage probabihty of this confidence interval and leaves its length unchanged. The 
upper endpoint of the Clopper-Pearson 1 — a confidence interval is 1 for y = n; otherwise 
it is the solution for 9 of 

Pe{Y <y) = Fe{y) = \. (3) 
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The lower endpoint of this interval is for ?/ = 0; otherwise it is the solution for 6 of 

Pe{Y >y) = l- Fe{y - 1) = f • (4) 

Convenient expressions for the solutions of these equations are described e.g. by Casella 
and Berger, (2002, p. 454). We denote the Clopper-Pearson interval by \tcp{Y), ucp(Y)\ . 

3. Randomized confidence interval for the binomial probability 

The randomized confidence interval of Stevens (1950) can be found by considering 
the artificial data Z = Y + V, where V and Y are independent random variables and 
V U{0, 1). Also assume that either V G [0, 1) or G (0, 1]. For observed values y, v 
and zofY,V and Z, respectively, this confidence interval for 6 is found as follows. The 
p- value for testing Hq : 6 = 6 against Ha : 6 < 6 is P§{Z < z). A 1 — a upper confidence 
interval for ^ is {^9 : Pe{Z < z) > a/2}. We replace all upper confidence intervals of the 
form [6, c) by [b,c\. This does not decrease the coverage probability of this confidence 
interval and leaves its length unchanged. The upper endpoint of the randomized interval 
is 1 for y = 72 and v > a/2 ; otherwise it is the solution for 6 of 

a 

Pe{Z <z)= vfeiy) + Fe{y - 1) = (1 - v)Fe{y - 1) + vFe{y) = -. (5) 

The lower endpoint of this interval is for ?/ = and v < 1 — a/2] otherwise it is the 
solution for 6 of 

Pe{Z >z) = {l- v)fg{y) + 1-Fg{y) = {l- v) (l - Fg{y - 1)) +v{l- Fgiy)) = -. (6) 

Denote the resulting confidence interval by [ifi{y,v),Ufi{y,v)~\. This interval satisfies 
both of the following conditions: 

Peie > URiY,V)) = - for all 6 
Pe{e < Ir{Y,V)) = - for all 9. 

By comparing the left-hand sides of ([5]) with ([3]) and ([6]) with (j4j), we find that for 
every y and v G [0, 1], this confidence interval is contained strictly within the Clopper- 
Pearson 1 — a confidence interval. The confidence interval lower endpoint ^R{y■, v) is a 
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nondecreasing function of v that is strictly increasing for (a) ?/ = and v > 1 — a/2 and 
(b) all y > 1. The confidence interval upper endpoint UR{y,v) is nondecreasing function 
of V that is strictly increasing for (a) y — n and v < a/2 and (b) all y < n — 1. 

The confidence interval [£r{Y,V),ur{Y,V)] dominates the 1 — a Clopper- Pearson 
confidence interval. This domination is possible because "losses for interval estimation 
and hypothesis testing are not usually convex" (Casella and Berger, 1999, p. 484). For 
the 1 — a Cloppcr-Pcarson confidence interval [icp(X)j'^Cp(X)]y Peip > Ucp{Y)^ and 
Pe(^9 < icp{Y)) are discontinuous functions that typically take values well above a/2 
for some values of ^. By contrast, the confidence interval [•^ii(y^, V),ur{Y, V)] has ideal 
coverage properties. The excellent theoretical properties of this randomized interval can 
be traced to the fact that the addition of F to y has "spht" each observation y into 
a continuous set of values, where the values that y is split into are less than all of the 
values that y + 1 is split into for each y = 0, . . . ,n — 1. In the language of Kabaila and 
Lloyd (2006), Z is a "refinement" of Y. 

As described in Appendix A, the confidence interval [iR{Y,V),UR{Y,V)] may be 
generalized by allowing the lower and upper endpoints to depend on different random 
variables and Vu, respectively, where each of these random variables is uniformly dis- 
tributed on (0, 1). However, as explained in Appendix A, there seems to be no advantage 
to be gained from this generalization. 

As described in Appendix B, we may also construct a randomized confidence interval 
for 9 using an auxiliary discrete random variable W. This random variable may be 
viewed as an approximation to V, which has a uniform distribution on (0, 1). 

The usual interpretation of the coverage probability of a confidence interval is that, 
in a sequence of independent repetitions of the statistical experiment that gave rise to 
this confidence interval, the long-run proportion of confidence intervals that includes 
the parameter is equal to the coverage probability. As described in Appendix C, this 
interpretation allows us to consider confidence intervals for 9 that are infiuenced by an 
appropriately-chosen auxiliary deterministic sequence, instead of the observed value of 
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an auxiliary random variable such as V. These deterministic sequences may be pseudo- 
random, quasi-random or possess a very obvious pattern. What we do in this appendix 
is to replace expectations by the corresponding long-run averages. 

4. Objections to the use of randomized confidence intervals in practice 

Cox and Hinkley (1974, p. 100) view randomization of this type as "a mathematical 
artifice" that is "of no direct practical importance". Two very cogent objections to the 
use of randomized confidence intervals in practice are the following: 

(1) Two scientists using the same procedure to construct a randomized 1 — a confidence 
interval for 9 based on the same observed value y will, with probability 1, produce 
different confidence intervals. 

(2) The randomized interval is influenced by an auxiliary random variable V that has 
no relation to the problem under consideration. 

These two reasons are presented, for example, by Kiefer (1987, p. 50) and Korn (1987, 
p. 707). Would the first of these objections be reduced if the following procedure were 
adopted? A website maintained by a reputable organisation would, upon the provision 
of the name of the user and the title of a project, provide an observation vofV^ U{0, 1) 
derived from a genuinely random source, such as electronic thermal noise. Together with 
this observation, this website would provide a identification number. The user would 
then use this observation v to construct his/her realisation of a randomized confidence 
interval and report this interval, together with v and this identification number. The 
website would permanently list the names of all users, projects, identification numbers 
and values of v. 

In Appendix C, wc show how confidence intervals depending on an appropriately- 
chosen auxiliary deterministic sequence have the desired long-run properties. Such a 
sequence may be pseudorandom, quasi-random or may be a sequence with a very obvious 
pattern. However, it would seem that the alarm experienced by practitioners in response 
to having their confidence interval being infiuenced by an auxiliary variable increases as 
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we move from random variable to preudorandom variable to quasi-random variable to a 
variable showing a very obvious pattern. 

We now add a third reason for rejecting the use of randomized confidence intervals 
in practice. Statisticians who believe that inference should be carried out conditional 
on an appropriate ancillary statistic (see e.g. Cox and Hinkley, 1974) would have the 
following objection to the use of such a randomized confidence interval in practice. The 
random variables Y and V can be recovered from the random variable Z. The statistic 
V has a distribution that does not depend on 6 i.e. it is an ancillary statistic. Carrying 
out inference conditional on \^ = f is equivalent to carrying out inference based solely 
on Y, leading to a non-randomized confidence interval. 

5. "Data-randomized" confidence interval for the binomial probability 

An apparent solution to the first of the objections described in the previous section 
and a mitigation of the second and third objections has been proposed by Korn (1987). 
This author defines W to be the one-sided p-value from the Wilcoxon rank-sum test 
for testing the null hypothesis that the ones in the sequence Xi, . . . ,Xn are randomly 
distributed in this sequence against the alternative hypothesis that they come near the 
beginning of this sequence. The distribution of W, conditional on Y = y, is uniform on 
{l/(p, 2/(p, . . . , |, so that it does not depend on 9. This conditional distribu- 

tion stochastically dominates the distribution of V ~ U{0, 1). Korn (1987) uses this to 
prove that ([7]), stated in Appendix B, holds true. 

Korn (1987) does not describe how the lower endpoint of his randomized confidence 
interval should be found. Based on the work presented in Appendix B, it is clear that the 
lower endpoint of this interval should be found as follows. Define the discrete random 
variable W by the requirement that, conditional on Y = y, W = W — Thus, 
conditional on Y = y,W is uniformly distributed on |0, l/(p , . . . , ( (y) ~ l) / (y) } • Hence 
the data-randomized confidence interval for 9 is [iR{Y,W),UR{Y,W)~\. This interval 
satisfies ([7j) and ([8]) (stated in Appendix B). If ri is not too small then ([9]) and (|T0|) (stated 
in Appendix B) are also satisfied and the expected length functions of the confidence 
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intervals [ifi{Y,V),U{f{Y,V)~\ and [^/{(F, W^), W^)] are approximately equal. It is 

straightforward to show that W),ur{Y, W)] dominates the 1 — a Clopper-Pearson 

confidence interval. This confidence interval eliminates the first of the objections raised 
in Section 4 since the data determines the randomization. Consequently, Korn (1987) 
calls these "data-randomized" confidence intervals. 

The excellent theoretical properties of this data-randomized interval can be traced 
to the fact that the addition of 14^ to F has "split" each observation y into (^*) values, 
where the values that y is split into are less than all of the values that y + 1 is split 
into for each y — 0, . . . ,n — 1. In the language of Kabaila and Lloyd (2006), Y + W is 
a "refinement" of Y. The upper endpoints of the data-randomized confidence intervals 
are based on Y + W, which can take 



possible values. The excellent theoretical properties of this data-randomized confidence 
interval suggest that it can be chosen as a standard against which other data-randomized 
confidence intervals can be judged. 

Of course, conditional on y = y, there are (^) equally-likely distinct locations of 
the y ones. Any one-to-one correspondence between these distinct locations and the 
integers 1, 2, . . . , (^) can be used, in the obvious way, to generate a random variable 
with the same conditional distribution as W. This random variable could be used as 
an alternative to W to construct a data-randomized confidence interval with the same 
coverage and expected length properties as [^_R(y, W),ur{Y, W)] . 

6. Objections to the use of data-randomized confidence intervals in practice 

Senn (2007ab) has objected to data-randomized inference procedures on two general 
grounds that specialise in the present circumstance to the following: 

(1) The "spht" of each observation y leads to quite an arbitrary ranking of the values 
into which y is split. Senn (2007a) says that any such split should be based only on 
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some meaningful comparison of the values that arise from a given observation y. In the 
present circumstance, there does not appear to be any meaningful comparison that could 
be used as the basis for this split. 

(2) The confidence interval described by Korn (1987) is only one of many possible data- 
randomized confidence intervals with the same theoretical properties. If W is replaced 
by W*, which is obtained by calculating W after the observations have undergone a 
given permutation then the resulting data-randomized confidence intervals have the same 
theoretical properties. Thus users of data-randomized confidence intervals will only be 
able to find a unique 1 — a confidence interval for given data xi, . . . , x„ if a convention 
can be established that the interval is based only on the auxiliary random variable W 
proposed by Korn (1987) (and not some alternative auxiliary random variable W* with 
similar properties). However, establishing such a convention does not seem realistic. 

Now it might be argued that the improvement in the properties of the confidence interval 
for 6 justifies the "splitting" of each observation y and that such a split does not require 
any meaningful comparison of the values that make up this split. However, even if a 
convention could be enforced that the data-randomized confidence intervals for 9 are 
based only on Korn's auxiliary random variable W , these confidence intervals would 
still not satisfy the invariance property described in Example 2.35 on Cox and Hinkley 



7. Confidence intervals for 6 based on splitting the Bernoulli data into two 
groups of approximately equal relatively prime size 

As before, suppose that Xi,X2, . . . ,X„ are independent and identically Bernoulli(6') 
distributed and that our aim is to find a confidence interval for 9 that satisfies ([T]) and 
02]). In this section, we consider the properties of a confidence interval for 9 that is 
obtained as follows. Suppose that n = rii +n2, where rii and n2 are relatively prime and 
as close as possible. Form the following estimator of 9: 



(1974). 
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where Yi = Xi + ■ ■ ■ + X^^ and Y2 = Xn^^i + ■ • • + X^- This is an unbiased estimator 
of 6. Consider the following procedure for finding a 1 — a confidence interval for 6. The 
Clopper-Pearson interval [icp{y):Ucp{y)] is the intersection of upper and lower l — a/2 
confidence intervals for 9 that are based on inverting the family of hypothesis tests using 
the test statistic Y (or, equivalently, the test statistic Y/n). We can construct an anal- 
ogous confidence interval that is the intersection of upper and lower l — a/2 confidence 
intervals for 6 that arc based on the test statistic 0. Let us denote this confidence inter- 
val by ti^(X)] . This confidence interval is obtained by deterministically splitting 
the data into two parts. A random splitting of the data into two parts is considered by 
Decrouez and Hall (2013b). 

For concreteness, consider the particular case that n — 47. Form the following 
estimator of 9: 



2 V23 24^ 

where Yi — Xi + h X23 and Y2 — X24 + h We have obtained Yi and Y2 by 

splitting the 47 Bernoulli trials into two groups of approximately equal relatively prime 
size. How do the confidence intervals [icp{Y),ucp{Y)~\ and [i'^{X),u\X)~\ compare? 
Wc expect the estimator to be a somewhat less efficient estimator of 9 than the 
maximum likelihood estimator Y/n. This is because we give the same weight to the 
estimators Yi/23 and Y2/2A, when the more accurate estimator 1^2/24 should have been 
given a larger weight. On the other hand, the estimator has 24 x 25 — 1 = 599 
possible values, whereas Y/n has only 48 possible values. We therefore expect that the 
Pe{9 < l\X)) and Pe{9 > u^{X)) will tend to be closer to a/2 than Pe{9 < icp{Y)) 
and Pe{9 > Ucp(Y)^, respectively (cf. Decrouez and Hall, 2013a). The fact that the 
estimator has many more possible values than the estimator Y/n can also be expected 
to lead to a shortening of the confidence intervals that will, to some extent, compensate 
or even overcome the inefficiency of the estimator by comparison with the estimator 
Y/n. 

Observe, however, that the confidence interval [£^(X), m^(X)] may be viewed as 
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a data-randomized confidence interval for 6. As suggested in Section 5, we use the 
data-randomized confidence interval of Korn (1987) as the standard against which we 
judge \p{X),u\X)\. We expect [^'''(X), ^'''(X)] to have coverage and expected length 
properties that are inferior to the data-randomized confidence interval of Korn (1987). 
For a start, the upper endpoints of the confidence intervals of Korn (1987) are based 
on a statistic that can take 2^^ fa 1.407 x 10^^ values. This is much larger than the 
599 possible values of the statistic 0, on which [£^'(X), w'^ (X)] is based. In addition, 
the statistic 6 orders the data in the wrong way. The statistic on which the confidence 
interval is based should always take a larger value for observed value y — t than for 
y — t. However, for yi — t and y2 — the observed value is y — t and 9 — t/A6, which 
exceeds 9 — {t + l)/AS when yi = and y2 = ^ + 1 (so that the observed value is y — t+1) 
when t > 23. Also, 9 takes the same value, 1/2, for {yi,y2) = (0,^2) (so that y = 112) 
and (1/1,^/2) = (^i)O) (so that y = ni). In the language of Kabaila and Lloyd (2006), 
is not a "refinement" of Y. The fact that orders the data in the wrong way may 
be interpreted as just another manifestation of the inefficiency of this estimator. This 
means that if we are prepared to consider data-randomized confidence intervals then we 
should be using the data-randomized confidence interval of Korn (1987) instead of the 
confidence interval [i'^{X),u\X)] based on 0. 

8. Discussion 

Various kinds of randomized, pseudorandomized and data-randomized "equi-tailed" 
confidence intervals for the binomial probability, based on iid Bernoulli observations, have 
been reviewed. Of course, randomization, pseudorandomization and data-randomization 
can be combined in various ways. For example, we could combine randomization with 
data-randomization. Undoubtedly, such confidence intervals will continue to be of theo- 
retical interest. 

The standard confidence interval that satisfies the "equi-tailed" coverage constraints 
described in the paper is the Clopper-Pearson interval, which is not randomized (or 
pseudorandomized or data-randomized). Broadening the class of allowable interval es- 
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timators to include cither randomization, pseudorandomization or data-randomization 
(or a combination of some of these) may be viewed as allowing one to use an additional 
resource. The theoretical question is: How well is this additional resource being used? 
We have asked and answered this question in the case of the unusual confidence interval 
for the binomial probability described in Section 7. Of course, whether randomized, 
pseudorandomized or data-randomized confidence intervals will ever be used in practice 
is open to question. 
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Appendix A: A generalization of the randomized confidence intervals 

A generalization of the randomized confidence interval ^iR(Y,V),Ufi{Y,V)'j is 
iJ'rO^,Vu)], where Y and (Vg, \4) are independent, Ve ~ [7(0, 1) and \4 ~ U{0, 1). One 
could, for example, choose — 1 — Vy,. This interval satisfies the following conditions: 

Pe{e>UR{Y,Vy)) foraU 6 

Pe{e < eRiY,Vi)) ^ - for all 9. 

Also, the confidence intervals [iR{Y,V),uii{Y,V)] and [iR{Y,Ve),uii{Y,Vu)] have the 
same expected length functions. There seems to be no advantage to be gained from this 
generalization. For example, suppose that Vj^ — 1 — V^. In this case, the confidence 
interval lower endpoint f^) = 1 — Vu) is a decreasing function of Vu for (a) 
y = and \ — Vu > 1 — a/2 and (b) all y > 1. This means that, for given observed 
value y, the main effect of increasing Vu is to widen the confidence interval. In statistical 
practice, confidence interval width is interpreted as a measure of the accuracy of the 
estimation of 6. It does not seem helpful to report (according to this interpretation) 
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varying apparent accuracies of estimation of (depending on the value of v^)-, for the 
same observed value y. 

Appendix B: Randomized confidence intervals that depend on an auxiliary 
discrete random variable 

Suppose that the random variable W is such that, conditional on y = y, 1^ is 
uniformly distributed on [l/M{y), 2/M{y), M{y)/M{y)}, where M{y) is an integer 
greater than 1 for each y = 0, . . . ,n. A particular case is that M{y) = M for y = 
0, . . . ,n. Define the discrete random variable W by the requirement that, conditional 
on Y — y,W — W — 1/M{y). Thus, conditional onY — y,W is uniformly distributed 
on {0, 1/M(y), . . ., (M(y) - l)/M{y)]. Let Fy, Fw and denote the cumulative 
distribution functions of F ~ C/(0, 1), W and W, respectively. Using the facts that Fw 
is stochastically larger than Fy and Fy is stochastically larger than F^^, it may be shown 
that the confidence interval [£ij(y, 1^), ^^(y, H^)] satisfies the following conditions: 

Pe{e > ur{Y,W)) <- for all 9 (7) 
Pe{e<£R{Y,W)) <- for all 6. (8) 

If the smallest of the M(y)'s is not too small then, in addition, 

Pe{e > uniY,W)) ^ - for all 9 (9) 
Pe{e < £r{Y, W)) ^ - for all 9 (10) 

and the expected length functions of the confidence intervals [iR{Y,V),UR(Y,V)~\ and 
[£ij(y, ly), Kij(y, H^)] are approximately equal. These results may be interpreted as 
resulting from the fact that W may be viewed as an approximation to V, which has a 
uniform distribution on (0, 1). 
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Appendix C: Confidence intervals that depend on an auxiliary deterministic 
sequence 

Suppose that ¥^,¥2, . . . are independent and identically Binomial(n, ^) distributed. 
In other words, suppose that we carry out independent repetitions of the statistical 
experiment that gives rise to ¥. Let f 1, f 2, . . . be a deterministic sequence of real numbers 
such that either Vk G [0, 1) for every k = 1,2, . . . or G (0, 1] for every k = 1,2, ... . 
Now suppose that Vi,V2, . . . is uniformly distributed modulo 1, as defined by Kuipers 
and Niederreiter (1974). For any given irrational number A, the sequence Vk — {nX}, 
where {a} denotes the fractional part of a, possesses these properties and may be viewed 
as a pseudorandom sequence. The van der Corput sequence (defined e.g. on page 127 
of Kuipers and Niederreiter, 1974) possesses these properties and may be viewed as a 
quasi-random sequence. Suppose that, given the observation of Y/j, we compute the 
confidence interval [£R{yk, Vk),UR{yk, Vk)] . We use the notation 

1 if ^ is false 

where A is an arbitrary statement. It may be shown that, for each 9, 

^ m 

— ^^X(6' > Mij(Ffc, Vfc)) converges almost surely to — and 
^ k=i 

^ rn 

— ^^X(6' < £i^(Yfe, Vfc)) converges almost surely to — 



m 

k=i 



as m — 00. It may also be shown that, similarly, the long-run average lengths of the 
confidence intervals [£r{¥,V),ur{¥,V)] and [iR{yk,Vk),UR{yk,Vk)] are the same. 

Alternatively, we may suppose that Wi,W2, . . . is a deterministic sequence of real num- 
bers such that Wi, W2, ... is a periodic sequence with period N, where {wi, W2, . . . , wn) 
is a permutation of {1/N,2/N, . . . , N/N). There are both pseudorandom sequences 
(found using e.g. mixed congruential generators with maximal possible cycle length 
A^) and sequences with very obvious pattern (e.g. 1/N ,2/N , . . . ,N /N ,1/N ,2/N , . . .) 
that satisfy these conditions. Define the sequence Wi,W2,--- by iD^ = w/; — 1/A for 
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k = 1,2, . . . . Suppose that, given the observation i/k of Y^, we compute the confidence 
interval [iR{yk,Wk),UR{yk,'Wk)]- It may be shown that, for each 9, 

— ^^X(6' > UR{Yk,Vk)) converges almost surely to a number < — and 
^ k=l 

^ m 

— ^^X(6' < iR{Yk,Vk)) converges almost surely to a number < — 

777- ^ 
k=l 

as m — )■ oo. 
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